Optical spectrometer with enhanced spectral resolution from an unregistered tristimulus detector

ABSTRACT

A spectrometer includes a spectrogram, digital camera and signal processing to compensate for limits of system spatial resolution, spatial distortions and lack of precision spatial registration, limited dynamic range, The spectrogram is captured by a digital camera, and the corresponding image is converted to a wavelength and magnitude with mitigation of optical point spread function and potential magnitude clipping due to over-exposure. The clipped portions of the signal are reconstructed using tangential adjacent point spread functions as a reference or adjacent channel ratios as reference. Multichannel camera detectors having unique response magnitude ratios per wavelength are exploited to make associated direct mappings, thereby making improvements in wavelength resolution and accuracy to up to at least one to two orders of magnitude.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 14/302,291, filed Jun. 11, 2014, entitled “OPTICALSPECTROMETER WITH ENHANCED SPECTRAL RESOLUTION FROM AN UNREGISTEREDTRISTIMULUS DETECTOR, the disclosure of which is incorporated herein byreference in its entirety.

TECHNICAL FIELD

This disclosure relates to optical spectrometer instrumentation, andmore particularly to a method of using a tristimulus detector common incommercially available cameras, including low cost consumer cameras andwebcams, without the need for high resolution optics nor precisionmechanical registration typically required from most spectrometers.Embodiments of the invention apply to general purpose spectrometry, withparticular advantages in applications with spectral lines, including,for example, Raman spectrometry and emissions spectrometry. In addition,embodiments of the invention increase effective dynamic range of thedetector by reconstructing clipped signals due to over exposure.

BACKGROUND

Applications of spectrometers, especially in applied spectrometry fordetermining chemical composition and deformulation, chemical properties,structural integrity and chemical verification against counterfeit ormishap, are of great interest to the pharmaceutical, forensics,biotechnology, food and agriculture, mining, mineralogy, gemology,petroleum exploration, medical diagnostics, electronics and otherindustries. For food and agriculture, spectrometry has been applied fordetermining the composition and ripeness of food as well as fordetecting contamination due to harmful chemical agents and pathogenssuch as infectious bacteria. Nutritional information of food ingredientsand food in solution may be determined using methods such as Ramanspectrometry, which uses a spectrometer to measure the shifts in light(visible and/or infrared) from a monochromatic excitation, typicallyfrom a laser, to other frequencies. In all applications, the wavelengthor wavenumber, the inverse of wavelength, are important. In some cases,for example for Raman spectrometry, the difference between theexcitation spectral line wavenumber and the measured spectral linewavenumbers are important.

FIG. 1 is a diagram that illustrates functional blocks and processingfor a conventional spectrometer. Sometimes spectrometers are also calledspectrographs or spectroscopes, and the terms are used interchangeablyin this description. Spectrometers, spectrographs, and spectroscopesproduce spectrograms, which are visual representations of spectralfrequencies. Sometimes an entire system is described as a spectrometer,with a spectrograph or spectroscope described as a component of thespectrometer system.

Conventional spectrometer technologies include an internal or externallight source 2, an optional specimen for determining absorption,transmission or re-emission 4, a spectroscope 6 that produces aspectrogram, a detector 8, processing for removing background signal andnoise 10, and optionally further normalization 12 for the cases ofabsorption and transmission measurement. FIG. 2 is a block diagramillustrating a conventional Raman spectrometer. As shown in 24 of FIG.2, the conventional spectroscope 6 of FIG. 1 typically has optionalfront end optics 26 and 28, a monochromator 30 (an optical dispersionspectral separator) such as an optical grating, prism or similarmechanism, optional second optics (sometimes combined with the detectoras in a camera 32), and an optically isolating housing which may or maynot include the entire detector.

Spectrometry, which in general is the application of spectrometers tostudy objects, typically requires a controlled light source, commonly alaser or broad band source. However, the spectrometer is often aseparate device and does not include the light source. The detectorwithin the spectrometer is generally sensitive across a broad band ofradiation frequencies. In some cases, tristimulus detectors have beenused, for example Charge Coupled Device (CCD) cameras with optical red(R), green (G) and blue (B) filters, but only for determining a singleintensity estimate along the position of the spectrogram. This intensityestimate is taken directly from a color image, which is comprised of red, green and blue (RGB) primaries. Since the intensity is associated onlywith spatial position along the spectrogram, the resulting spectralresolution is limited to the resolution of the optics. The resolution ofthe optics is typically primarily determined by the width of the slitopening where the light enters the spectrometer.

For example, the Rspec Explorer is a relatively inexpensive commerciallyavailable spectrometer, available from fieldtestedsystems.com. Its USBcamera is housed in a black box which also includes a diffractiongrating and lens. A separate pair of adjustable black foamboard panelsare supplied to provide an optical slit, which may be a few feet orfurther away from the black box. This external slit determines the limitof the optical resolution and therefore also the spectral resolution. Anarrow slit improves resolution, but limits the light level for thespectrometer. Thus there is the classic trade-off between spectralresolution and signal to noise ratio. The USB camera is an RGB based CCDcamera that captures the conventional image of the camera, with thediffraction grating image super-imposed on the right side. Software thatruns on a PC takes this image and creates magnitudes from the RGB imageof the spectrogram.

The determination of measured wavelengths or wave numbers from thetypical spectrometer are generally determined by spatial location, inturn determined by design, requiring strict adherence to particularalignment of all components in the optical path, and usually furtherrefined through calibration. The difficulty of alignment has beenmitigated in many cases by using lenses and filters in two directions,with beam splitters and other optical components that tend to be lossy,that sometimes causes marginal signal-to-noise ratios.

In the Rspec Explorer example, the user must align the peak of the slitfrom the direct camera image to a reference line (graticule) shown in awindow of the corresponding software application on the computer. Thealignment is typically somewhat tedious and imprecise as the peak isoften too broad due to the slit being too wide. If instead the slit isnarrow enough for precise alignment, the resulting spectrum magnitude istypically near or below the noise floor of the camera, or the referencepeak is so high that it is beyond the dynamic range of the camera,resulting in clipping. This clipping means limiting the peak to themaximum camera digital code for amplitude in the respective channel(s).So, as with other spectrometers, the wavelength resolution of the RspecExplorer is no better than the optical resolution determined by thecombined point spread function of a slit, the resolution of the CCD, andthe intermediate optics. And, as with other spectrometers, because ofthe trade-off between light intensity and resolution due to slit width,the camera's CCD dynamic range, determined by sensitivity and noisefloor, also factors into the determination of the optical resolution andthus the spectral resolution.

Another type of existing spectrometer is one that uses a simple very lowcost spectrogram and a web cam detector. Many of the key performanceissues with this prior art is summarized in Kong Man Seng, “Trace gasmeasurement using a web cam spectrometer,” March 2011, City Universityof Hong Kong, Department of Physics and Materials Science. The“Discussion” of section 5, page 33-40, discusses the typical issues withslits, alignment, optics, noise and similar issues that cause limitedoptical resolution and dynamic range. This same section includes some ofthe typical methods for mitigating these issues, the vast majority ofwhich depend on improving optical resolution directly, improvingalignment mechanically and increasing light source power along with heatand other energy management required to prevent damage to components dueto the increased radiation.

In general, typical methods for improving spectrometer accuracy havebeen to increase optical resolution and precision, increase precision ofoverall mechanical and optical alignment, increase light power, and toincrease detector dynamic range. Each of these increases cost with everdiminishing returns of improvement. Detector dynamic range is typicallyimproved by reducing the noise floor and allowing for integrating steadystate or repeated signals over time. The most common detector uses CCDtechnology. For highest dynamic range, the CCD is cryogenically cooledto mitigate one form of noise, while other forms of noise are stillpresent. The noise types can be rebalanced by custom CCD design, thusimproving cryogenic performance. The resulting detector system may beorders of magnitude more expensive than one based on consumer cameratechnology.

Many alternative designs attempt to mitigate issues associated with theloss of light through filters, narrow slits, etc., typically by addingmore expensive optical components, light sources and the like.

Further, depending on the specific optical arrangement, often thewavelength as a function of distance along the spectrogram primary(frequency) axis is non-linear and follows a cosine function. Cosinecorrection is optionally included, often complicating the design.

These improvements in accuracy add significant expense. Many of themethods to improve accuracy, especially in combination, also cause theinstruments to be large, bulky and prevent portability, or requireadditional significant expense to reduce size. Also, when integration isrequired to compensate for weak signals due to small slit size, thestability of the light source becomes critical, thus increasing cost,size and complexity of the light source. For the extra electronics andlight power, the power supply required becomes significantly larger andmore expensive.

Embodiments of the invention address these and other limitations of theprior art.

SUMMARY

Embodiments of the invention are directed toward a simple, inexpensivemethod that does not require bulky, high power components nor suchprecision optical resolution nor careful alignment of the optical path,allowing for the detector to be more arbitrarily placed relative to thespectrogram image, and allowing for adaptation to spatial shifts, tilts,geometric distortions and other results of misalignment. This not onlysimplifies the rest of the design, but also allows for the detector tobe separate from the spectrogram. Thus, using embodiments of theinvention, the detector can be the camera on a smart phone, personalcomputer, webcam or consumer stand-alone electronic camera, hand-held ormounted, to digitally capture the spectrogram image. The remainingcomponents for creating the spectrogram may be as few as the spectralseparation component, such as a diffraction grating, the housing and alight source (for some applications such as gemology, an external, evennatural light source). It is also particularly desirable to not requirethe optical resolution to determine the spectral resolution, especiallyfor spectral lines as is inherent in Raman spectroscopy and emissionsspectroscopy.

Accordingly, embodiments of the invention provide an opticalspectrometer with significant improvements in accuracy for both simple,inexpensive components and systems as well as for more expensive andhigher precision components and systems. Many applications formerlyrequiring expensive precision components may now be replaced with aspectrometer system with liberal spatial alignment of many principalcomponents, and in particular with potentially very liberal spatialalignment of the spectrogram with the detector. For spectral lines, theoptical resolution may be orders of magnitude worse for the sameresulting spectral resolution. In one embodiment, the method uses arelatively low cost electronic camera that produces a tristimulus imageof red, green and blue (RGB) samples per pixel. These channels withdifferent wavelength sensitivities are used to determine wavelengthmostly independently from the optical resolution. The dynamic range isless limited due to compensation for clipping.

For transmission, reflection and/or absorption, that typically requiresa reference broadband light signal (for example, white light), thereference spectrogram is captured as RGB, converted to magnitude, andthe wavelength of each spatial location of the spectrogram within theimage is determined algorithmically. Subsequently, measurements may bemade with magnitude relative to the reference for each wavelengthaccording to spatial location in the RGB image.

For emissions, including those from burning, Raman scattering andsimilar line spectra, no reference broadband spectrogram need becaptured. Generally, the relative line spectra magnitudes are used foranalysis, including chemical identification and principal componentanalysis. For all embodiments, the spectral resolution and accuracy areindependent from optical resolution except in the case where two or moreadjacent spectral lines are close enough to be within a significantportion of the optical point spread function. In other words, forexample, if a green line is smeared due to poor optical resolution, itis still a green line with unique wavelength determined by the methodherein described. If a green line and a slightly more yellowish greenline are blurred into each other by poor optical resolution then theresulting accuracy improvement using the method described herein may notbe as significant. In many significant applications, this requirement isnot a limitation.

The objects, advantages and other novel features of the invention areapparent from the following detailed description when read inconjunction with the appended claims and attached drawing views.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a generic block diagram view of a conventional spectrometerand process flow.

FIG. 2 is a generic block diagram view of a conventional Ramanspectrometer and process flow according to the prior art.

FIG. 3 is a generic block diagram illustrating components and processflow of a simple, inexpensive, relatively accurate spectrometeraccording to embodiments of the invention.

FIG. 4 is a block diagram view of methods of nonlinearity compensationaccording to embodiments of the invention.

FIG. 5 is a block diagram of method of the mitigation of clipping due toover-exposure according to embodiments of the invention.

FIG. 6 is a block diagram of a method of patching a clipped channelaccording to embodiments of the invention.

FIG. 7 is a plot of an example extent of the clipped column vs. column(primary spectrogram axis) and for each of the red, green and bluechannels made by using processing techniques according to embodiments ofthe invention.

FIG. 8 is a plot of an example of the start and end of the clippedregion within the respective column for the red channel made by usingprocessing techniques according to embodiments of the invention.

FIG. 9 is a plot of a reference magnitude profile across a column in aportion of the spectrogram adjacent to a clipped region made by usingprocessing techniques according to embodiments of the invention.

FIG. 10 is a plot of the non-clipped (not over-exposed) portions of anexample red channel and the corresponding portions of the referencemagnitude profile made by using processing techniques according toembodiments of the invention.

FIG. 11 is a plot of the non-clipped (not over-exposed) portions of anexample red channel and the corresponding scaled portions of thereference magnitude profile made by using processing techniquesaccording to embodiments of the invention.

FIG. 12 is a plot of the original red clipped (over-exposed) column ofan example red channel and the corresponding scaled reference columnmagnitude made by using processing techniques according to embodimentsof the invention.

FIG. 13 is a plot of the patched red column of an example red channeland the corresponding scaled reference column magnitude made by usingprocessing techniques according to embodiments of the invention.

FIG. 14 is a plot of the patched red column of an example red channeland the original red clipped (over-exposed) column of an example redchannel made by using processing techniques according to embodiments ofthe invention.

FIG. 15 is a plot of the column centroids along spectrogram primary axismade by using processing techniques according to embodiments of theinvention made by using processing techniques according to embodimentsof the invention.

FIG. 16 is a plot of mean (R,G,B) along spectrogram secondary axis madeby using processing techniques according to embodiments of theinvention.

FIG. 17 is a plot in CIE 1931 xy plane depicting the conversion of xycoordinates to wavelength via projection from the reference white pointthrough the sample point to the pure monochromatic light curve made byusing processing techniques according to embodiments of the invention.

FIG. 18 is a plot of purity vs spectrogram primary axis made by usingprocessing techniques according to embodiments of the invention.

FIG. 19 includes plots of A) the measured wavelength, measWlen(n) 250,of an example wide band source and B) the theoretical wavelength,theoryWlen(n) 252, of an example wide-band light source vs. the primaryspatial axis of a spectrogram made by using processing techniquesaccording to embodiments of the invention.

FIG. 20 is a plot of an example of a green spectral line spectrogramwith poor optical resolution that may be addressed by using processingtechniques according to embodiments of the invention.

FIG. 21 shows the same main green spectral line ploted as maximummagnitude at the wavelength within the spatial vicinity of thetheoretical location for the wavelength made by using processingtechniques according to embodiments of the invention.

FIG. 22 is a functional block diagram illustrating stimulus control of alight or electromagnetic source controlled through use of a camera flashaccording to embodiments of the invention.

FIG. 23 is a functional block diagram of an example spectrogram systemaccording to embodiments of the invention.

DETAILED DESCRIPTION

As described below, embodiments of the invention make significantimprovements in measuring wavelength and magnitude from spectrogramimages captured using relatively inexpensive tristimulus detectors. Suchdetectors are widely available as stand-alone RGB cameras, embedded inmobile devices, such as smart phones (iPhone, Andriod, Blackberry), cellphones, notepads (iPad, etc.), laptops and other portable computers, andas accessories to computers such as USB cameras (webcams, inmicroscopes, telescopes, etc.). Many of these detectors includeprocessors on which particular operations may be performed, described indetail below.

In addition, embodiments of the invention improve the effective spectralresolution beyond the limits of the system optical resolution not onlydue to the optical limits of the apparatus for capturing the image ofthe spectrogram, but also beyond the optical limits of the spectrogrambeing captured. The amplitude measurement improvement includes bothnoise mitigation and non-linear distortion mitigation. The noisemitigation is achieved from both temporal and spatial integration of theappropriate wavelength. The non-linearity mitigation includesreconstructing peaks that have been clipped due to over-exposure.Together, these improvements in magnitude dynamic range can be over anorder of magnitude. For sufficiently separated line spectra, theresulting improvement in spectral resolution and accuracy can be ordersof magnitude.

Referring now to FIG. 3, first a spectrogram image 40 is detected andcaptured using a camera 42 or other equivalent detection and capturesystem. Embodiments of the invention takes advantage of low cost andincreasingly accurate CCD camera technologies that generally have red,green and blue channels per pixel. However, fundamentally other detectorarrangements may be used as long as for each wavelength detected, thereare unique ratios of the 2 or more sensors. For example, for the regionbetween green and red, each wavelength of the spectrum produces uniqueratios of the G and R channel responses. In an alternative embodiment,this unique ratio scheme may be extended down to the infra-red, IR,region and into the ultra-violet, UV, region of the commerciallyavailable (CCD) camera by eliminating the corresponding optical filter.Thus, mapping the wavelength to the corresponding ratios, the followingprocessing may be adapted to any number of color or frequency responsechannels using the same principals and the same processing blocks,including a wide spectrum IR-VIS-UV spectrogram. The visible spectrum,an RGB image, is the nominal embodiment, preferred for correspondingapplications. The description of the RGB case is sufficient for thoseversed in the arts to produce other embodiments tuned to other frequencybands using the same methods for mitigating optical resolution anddynamic range using the methods of this invention. The RGB image of thespectrogram is converted to magnitude and wavelength as described below.

Processors running on the image capture device or on a spectrometer mayperform processing by running operations in software running on suchprocessors. In some embodiments functions or operations may beprogrammed onto an FPGA or other firmware or hardware.

The RGB spectrogram image is optionally cropped in operation 44 toremove portions of the image that surround the spectrogram, therebyreducing the amount of pixels to process, for speed and/or reducedcomputation. For a nominally dark surround, cropping is performed byeliminating each line at the top and bottom, and each column on theright and left where all pixels are below a useful amplitude thresholdcorresponding to a noise floor or black. In a preferred embodiment, thecropped spectrogram has a small border of black sufficient to measurethe noise floor on both sides, top and bottom. Alternatively, thecropping may be performed by removing the portions of the image that donot correlate well with the relatively saturated colors in order as isexpected with a spectrogram. The cropped result is an image with mostlypure colors or black, with colors changing along the primary axis andcolors being relatively constant, but with varying intensity, along thesecondary axis. In an alternative embodiment, rotation of thespectrogram image is performed before or after cropping such that theprimary axis is parallel to image rows or lines, and the secondary axisis parallel to the image columns.

Next, two types of nonlinearity of the spectrogram are compensated in anoperation 46 as shown in more detail in FIGS. 4 and 5. Each channel, R,G and B, is converted in an operation 60 to “linear” light through thegamma portion of the reverse transform from CIE1931 xy coordinates asdefined by the respective colorimetry specification. Then any clippingdue to over-exposure is compensated in an operation 65 by estimating theclipped signal. Details of both of these methods follow.

As per most digitally encoded images, a gamma power function is used. Soin order to apply linear operations such as integration, scaling, etc.to each channel, the inverse of the gamma power function must be firstapplied. For example, for sRGB, the linear representations, Rlinear,Glinear and Blinear are calculated according to well known techniques asfollows:

-   -   If Rlinear<=0.03928    -   Rlinear=R+12.92    -   else    -   Rlinear=((R+0.055)/1.055)^(2.4)    -   If Glinear<=0.03928    -   Glinear=G+12.92    -   else    -   Glinear=((G+0.055)/1.055)^(2.4)    -   If Blinear<=0.03928    -   Blinear=B+12.92    -   else    -   Blinear=((B+0.055)/1.055)^(2.4)

Next, any clipping is mitigated as shown in FIG. 5. The {Rlinear,Glinear, Blinear} spectrogram image 70, with primary axis x 74 along thechange of wavelength and secondary axis y 76, perpendicular to theprimary, is checked for clipping due to over-exposure. In oneembodiment, the image may be further reduced for processing clipping byignoring rows and columns below an offset (nominally 6 dB) above thenoise floor, thereby resulting in a subimage 72. Any portion of theimage with channel values above a threshold near or equal to the maximumpossible value is considered clipped. For illustration, example of arectanglular patch of the spectrogram image containing the clippedportion of the red channel of a broadband spectrogram is illustrated inFIG. 5 as 82.

If any portion of a channel is clipped, clipping is located for each xlocation. In other words, for each location along the principal (x)axis, the locations along the secondary (y) axis of the start,clipStart(x) and end, clipEnd(x), of clipping are saved. For example,FIG. 7 shows the portion of each column at location x clipped for thespectrogram of a bright light bulb, a broadband light source that was sobright that it caused considerable clipping in all three channels. Forthe Rlinear, Glinear and Blinear channels, FIG. 7 shows the number ofsamples clipped along the y (secondary spectrogram) axis for each pointalong the x (primary spectrogram) axis for each channel respectively:Rclipped(x) 114, Gclipped(x) 112, Bclipped(x) 110. FIG. 8 shows a plotof the RclipStart(x) 120 and RclipEnd(x) 122, the start and end ofclipping respectively for the Rlinear channel, vs. x, the primary(column) spectrogram axis.

Referring again to FIG. 5, after this process of identifying andlocating any clipped portion 82 of the signal for each channel, acorresponding clip mitigation reference signal is located. There are twoprincipal methods of the clip mitigation reference signal. Both clipmitigation methods use a reference signal. The greater of the tworeference signal corresponds to one of the respective preferred methods.

The two principal methods of clip mitigation are: A) an adjacentunclipped column or mean of consecutive unclipped columns adjacent toand within the same channel of the clipped portion 78, or B) the meanratios of the unclipped portion of the set of Rlinear, Glinear andBlinear channels of the top and bottom portion of the clipped column 84.For the second embodiment, as an example, for each column of pixels in82, the mean triplet {Rlinear, Glinear, Blinear} is calculated for thesame column (same x value) in the combination of above 94 and below 84the clipped portion 82. In other words the mean of nearby unclippedimage segments is calculated for each channel, Rmuc, Gmuc, Bmuc. Then,for portions of the image where only Rlinear is clipped within 82, andat least one other channel is not clipped, the larger unclipped channelis the reference channel and the corresponding column is used as thelocal reference column within 82. Then the portion of the clippedRlinear signal within 82 is replaced with the scaled portion of thelocal reference column scaled by the ratio of the mean reference channel(Gmuc or Bmuc) with Rmuc. For example, for a given column x with clippedRlinear(x) within 82, if Glinear(x) is the only unclipped channel or ifit is larger than Blinear(x), then the clipped portion of Rlinear(x,y)is replaced with Glinear(x,y)*Rmuc(x)/Gmuc(x). Let the scale factor

-   -   s=Rmuc(x)/Gmuc(x)        and the reference column for a given x be given by    -   refColumn(y)=Glinear(y)

FIG. 12 shows an example of scaled refColumn 160 and the clippedRcolumn(y)=Rlinear(y) 162 vs. row y. FIG. 13 shows the patched versionof Rlinear, Rpatched 170 and the scaled refColumn 172 vs. y. FIG. 14shows the original clipped Rlinear =Rcolumn 180 and Rpatched 182 vs y.

For the first embodiment, the same strategy of replacing a clippedsignal with a ratio scaled nearby reference signal is applied. However,instead of referencing a different unclipped channel, the columnsegments adjacent (that is adjacent along the secondary y axis, to thetop 94 and bottom 84 of 82 in FIG. 5) to the unclipped portion of thesame clipped channel are used. These adjacent segments of the columnsare the skirts of the point spread function. The strategy is to find thescale factor for minimum mean square error between the skirts of theclipped column and the unclipped reference column. Then this same scalefactor is applied to the unclipped portion of the reference column toproduce a patch to replace the clipped portion. For applications wherenoise filtering is required, multiple columns within each region areaveraged.

An example method of matching skirts is as follows. As shown in theblock diagram of FIG. 6, first the indices of the clipped regions aredetermined in operation 100. As with the first embodiment of clipmitigation, this corresponds to the region contained within 82 of FIG.5, the number of clipped pixels 110, 112 and 114 vs x in FIG. 7 and thestart 120 and end 122 of clipping in FIG. 8. Again referring to FIG. 6,an average column curve below the clipped region is averaged andconcatenated with the average column curve above the clipped region inoperations 102, 104 and 106. This assumes that

-   -   A) the optical point spread function guarantees a non-clipped        samples of the digital image at the boundaries of the clipped        portion of the image (i.e. 94 and 84 are above 0 and not        clipped) and    -   B) the intensity profile across the spectrum naturally includes        some unclipped portion (i.e. 78 is above 0 and not clipped).

Using the same example of FIGS. 7 and 8, FIG. 9 shows a plot 130 ofrefColumn(y), the average of a few unclipped columns 78 (FIG. 5)adjacent to the clipped portion of the red channel, with a vertical line132 near the middle indicating the location of the correspondingcentroid.

The respective centroids are used for registration between respectivereference unclipped and clipped columns. The result is shown in FIG. 10:concatenated column segments, that is, the unclipped portions (fromskirt patches 94 and 84 of FIG. 5) of the columns of the clipped patch82, RColSegs(y′) 140 and the corresponding concatenated column segments,that is, portions (from skirt patches 96 and 80, respectively) of thereference 130 of this same channel, refColSegs(y′) 142. The scale, s, ofthe reference used to best match these unclipped segments is determinedusing a least mean squared error method. Preferred embodiments use thefollowing minimum mean least squared error method:

s=(refColSegs^(T)*refColSegs)⁻¹*refColSegs^(T)*RcolSegs

where refColSegs and RcolSegs are both N×1 column vectors.

Shown in FIG. 11 is the resulting rescaled concatenated reference columnsegments 152 as well as respective concatenated column segments of theclipped column 150.

Now applying this scale, s, also to the original portion of thereference 78, that is the portion corresponding to the clipped portionof Rlinear 82, we obtain a patch for and an estimate of the portion inRlinear that was clipped. The result 160 is shown in FIG. 12, along withthe original clipped column of Rlinear, 162. To further illustrate, thisportion of the scaled reference column replaces the clipped portion ofRlinear to create a patched column Rpatched, 170, as shown in FIG. 13,along with the full scaled reference column signal, s*refColumn 172.

FIG. 14 shows the original clipped Rlinear column, 180 and thecorresponding patched Rlinear column, 182.

Next a single value mean for each column is calculated for each channel.The mean value increases resolution of the relative magnitudes andreduces noise. Referring again to FIG. 3, for the corresponding stepoperation 48, the centroids each of Rlinear, Glinear and Blinear arefound for each column. The aggregate of these centroid points create acurve that approximates a line traversing the spectrogram. An examplecurve for a slightly rotated and uncropped spectrogram image is shownplotted as 190 in FIG. 15.

Next, in operation 50 shown in FIG. 3, the mean column value centered onthe respective centroid is taken. For each channel and each column,summation is performed centered on the respective centroid and includingthe portions from half magnitude on the bottom to half magnitude on thetop. For each column, this summation is divided by the total samplessummed within the column to calculate the mean value centered on thecentroid. FIG. 16 shows the corresponding mean magnitudes for Rlinear,Glinear and Blinear, that is, Rm as 200, Gm as 202 and Bm as 204,respectively.

The tristimulus set {Rm(x),Gm(x),Bm(x)} is then converted to magnitude,saturation and wavelength. The tristimulus set is converted towavelength in steps of A) converting to coordinates in a color plane, B)projecting the color plane coordinates to coordinates of purest form inthe color plane and C) selecting the wavelength whose coordinates comeclosest to those of the projection in step 2. One embodiment uses thevery commonly used pseudo-physiological CIE1931 xy color plane.

So as not to confuse the spectrogram primary axis x, with the x of thecolor plane coordinate system, the remaining text of the inventiondetails will substitute the spectrogram primary axis index variable xwith n, as in {Rm(n), Gm(n), Bm(n)}. Thus the corresponding CIE1931{x,y} values are {x(n),y(n)}.

The conversion of {Rm,Gm,Bm}to magnitude, saturation and wavelength isperformed as follows. First, following each of the {Rm,Gm,Bm} valuesalong the centroid curve are converted to CIE1931 {x,y} values using therespective colorimetry conversion operation 52 of FIG. 3. For example,most modern electronic cameras use sRGB (equivalent to BT 709 highdefinition television colorimetry). Details of this conversion fromCIE1931 xy to wavelength and saturation, as in operation 54 of FIG. 3,are as follows.

As depicted by the marked up plots of FIG. 17, the CIE1931 {x,y} valuesare projected to the curve corresponding to the most pure light of agiven wavelength, lambda. The projection entails the generation of aline, 220, from the reference white, {xw,yw}, 222, through {x,y}, 224,and continuing to the pure curve, 226, and intersecting at 228. Thewavelength corresponding to intersection 228 is the wavelength of thepure version of the {x,y} values. This process is performed for allvalues of each {Rm,Gm,Bm} along the spectrometer primary axis, withcorresponding points in the xy plane in FIG. 17 shown by the respectivecurve for an example broadband light source, 230.

The slope of the projected line 220 of FIG. 17 is given by

slope(n)=(y(n)−yw)/(x(n)−xw)

where {xw,yw} are the CIE 1931 coordinates for the reference white pointfor the camera colorimetry. In the case of sRGB, {xw,yw}={0.3127,0.3290}.

The corresponding angle with the horizontal (x) axis of FIG. 17 is given(using matlab, octave and scilab script language) by

if x(n) < xw  angleOffset = 3.1415927; else  angleOffset = 0; endangle(n) = atan(slope(n) ) + angleOffset;

Then angle(n) is matched to the angle in a table. The table has a columneach for angles, x coordinates, y coordinates and wavelength. The anglesare calculated from the arc tangent, atan, of the slope of the linebetween { xw, yw} and the respective { x,y} coordinates of puremonochromatic light of the given wavelength of each row of the table.The CIE 1931 {x,y} coordinate and wavelength data for the puremonochromatic light curve is given by Table 1 of section 3.3.1 of GunterWyszecki, W. S. Stiles, “Color Science: Concepts and Methods,Quantitative Data and Formulas, 2nd Edition,” 1982, John Wiley & Sons,NY, that is hereby incorporated by reference herein. The table withprecalculated angles from the CIE 1931 {x,y} coordinates is used forexpediency for converting angle to wavelength.

Thus, each {Rm,Gm,Bm} is converted to CIE 1931 {x,y} and projected topure light 226. The corresponding wavelength, lambda, given by theaforementioned reference table is used. In an alternative embodiment,linear interpolation between corresponding angles in the table may beused to determine wavelength at finer resolution.

Again referring to FIG. 17, a measure of purity of light that can beused to determine the appropriateness of treating the respective{Rm,Gm,Bm} set as a pure spectral line vs. broad-band. For an estimateof this purity, the ratio of the CIE vector length of the measuredsample (from 222 to 224) to the vector length of the corresponding purespectral line (from 222 to 228) is used. In an alternative embodiment,relative tristimulus purity is instead calculated as the ratio of theCIE vector length of the measured sample (from 222 to 230) to the vectorlength of the corresponding saturated tristimulus triangle (with avertex at each of the R, G and B coordinates). Thus, in someembodiments, each spectrogram pixel in a multichannel spectrogram imagemay be converted to respective wavelength,

FIG. 18 shows a plot of the purity estimation 240 using the tristimuluspurity method, vs sample along the centroid curve (along the primaryaxis).

In one embodiment, the purity estimation values are used to establishnominal mapping between the spectrogram primary axis n and the remainingwavelengths. In typical spectrogram designs, there may be a non-linearrelationship between spatial offset and wavelength. Cosine correction isoften included to compensate. For the case where a direct image captureof the spectrogram is taken, such compensation may need to be performedthrough image processing. The lambda values with the highest respectivepurity estimation values are used to established reference (control)points for registering the corresponding portions of the cosineuncorrected spectrogram, and then the remaining wavelengths followcosine correction established using known methods. Typically the best ofthe purest wavelengths to use for this purpose are first near yellow,where red and green channels are equal and second near cyan, where greenand blue are equal. These two wavelength points in the spectrum tend tobe especially useful for this purpose because A) the points where redand green are equal tend to be at mid-range sensitivities for twochannels, where they are less likely to suffer from low signal-to-noiseratio, nor from clipping or other high level related distortion and B)the human vision system is particularly sensitive to wavelengthdifferences near yellow and cyan, where the corresponding cones havehigh derivatives of sensitivity with respect to wavelength, and thus forcommercial success, cameras must be particularly accurate in theseregions. Yellow is better than cyan because in a typical colorimetry(including the example sRGB) yellow is fairly well saturated for thecase where channels are R=G, B=0 (the yellow point on the red to greenprimary line within the CIE 1931 xy plane), whereas the correspondingcyan line, G=B, R=0 (cyan point on the green to blue line within the CIE1931 xy plane) is not as saturated and the human eye is slightly lesssensitive to the change in wavelength. The human eye is much lesssensitive to changes in wavelengths at extremes of the visual spectra aswell as in the middle of green.

Accordingly, an example of measured wavelength, measWlen(n) as 250, andcosine corrected theoretical mapping of measured wavelength,theoryWlen(n) as 252, using yellow and cyan measured points is shown inplots vs n in FIG. 19.

The RGB values are also converted to a magnitude:

magnitude(n)=|R(n),G(n),B(n)|=sqrt(R(n)²+G(n)²+B(n)²)

Next, since wavelength typically is not a linear function of n, and aspectrometer produces magnitude vs wavelength, the next step is todetermine magnitude as a function of wavelength. Note that limits inoptical resolution, optical blur, cause a single essentially purewavelength of light to be spread, and thus measured, across a span ofthe spectrogram primary axis n. For example, for the case where a singlespectral line is alone in the spectrogram, the optical point spreadfunction of the system will spread this wavelength of light spatially.Most applications are especially interested in wavelength and magnitude,typically with particular value given to magnitude peaks, and noparticular value given to information to be gleaned from the opticalpoint spread function. Accordingly, for each wavelength, many magnitudesmay be measured across n. Among these many magnitudes for a givenwavelength, magnitudes measured far from the expected location (afterregistration above) of the spectrogram are generally ignored since theyare likely stray light or in some other way erroneous. Of the remainingmagnitudes measured for the given wavelength, the maximum is taken forthat wavelength. Thus the maximum magnitude within the vicinity of thetheoretical location (once mapped accordingly to the above method) isused as the measured magnitude for a given wavelength(n).

-   -   If (measWlen(n1)==measWlen(n2)) and (|n1−n2|<ndiffMax)    -   then mag(measWlen(n1))=max(magnitude(n1),magnitude(n2))        where measWlen is the measured wavelength and    -   ndiffMax corresponds to the expected optical point spread        function window width in sample units n.

For spectral lines this is a preferred embodiment. For broadbandspectra, the measured magnitude vs. theoretical wavelength is apreferred embodiment. In the preferred embodiments, the purity estimatevalues are used to cross-fade between these two methods of determiningmagnitude for a given wavelength.

FIG. 20 shows plots derived from an example of a green spectral linespectrogram with poor optical resolution. The curve 260 is a plot of themagnitude(n) vs theoretical wavelength, theoryWlen(n) according totheoretical position n for wavelengths after spectrogram registration.Thus, the poor optical resolution creates error in measurement ofmagnitude vs wavelength. In contrast, curve 262 is a plot of themeasured magnitude vs. measured wavelength, measWlen(n) using techniquesaccording to embodiments of the invention, thus showing greatly improvedspectral resolution and even improved accuracy of the wavelengthcorresponding to the peak magnitude and location. Note that 262 alsoshows many magnitudes for essentially a single wavelength.

FIG. 21 shows the same example green spectral line plotted asmaxVicMag(n), 270, the maximum magnitude at the wavelength within thespatial vicinity of the theoretical location for the wavelength.Comparing FIG. 21 with FIG. 20 shows that the spectral resolution hasbeen further enhanced. A small amount of red was also present as shownas 272. In an alternative embodiment, this spectral line may further beenhanced by taking the centroid for sub-sample resolution of wavelength.

Embodiments of the invention may be used to make devices such as: A)smart phone visible spectrometer B) smart phone Raman spectrometer andC) an infra-red (IR) spectrometer from a commercially availableelectronic camera with altered optical filters. In one embodiment, aRaman spectrometer includes a smart phone, a means of attaching andaligning the smart phone to the spectrogram housing such as a bracket orholder, a Raman excitation laser and a photo-detector trigger of thelaser. The laser is turned on when the photo-detector senses the smartphone flash. In these two examples, the imaging device and spectrometermay be either mounted or not mounted.

Embodiments of these may have spectrometer stimulus control via a cameraflash as shown in FIG. 22. The camera flash device 280 creates visiblelight detected by the respective detector 284, which in turn signals thespectrometer stimulus output control 288, thereby controlling thespectrometer electromagnetic radiation stimulus generator 292. Theaggregate response time of the flash, detector, output control andstimulus generation is sufficiently low to enable image capture by thecamera.

FIG. 23 is a functional block diagram of an example spectrometer system300 according to embodiments of the invention. In some embodiments someof these components may be combined or separated. The system 300 of FIG.23 includes a spectrogram 330 for reading electromagnetic energy from aspecimen 340, as well as a light source 332 for providing light orelectromagnetic energy directed to the specimen 340. A camera 310 has alens aligned to the spectrogram 330 for reading an output of thespectrogram. The camera 310 includes a flash 312, which may be used as asignal generator. A signal or flash detector 334 is coupled to the lightsource 332. In operation, when the camera 310 generates a light signalfrom the flash 312, the flash detector 334 may detect the flash as asignal to cause the light source 332 or energy source to generate itsown signal to illuminate the specimen so light or other energy may bedetected at the specimen by the spectrogram 330. A light guard 350shields the specimen from receiving light from the flash 312, whichallows the desired energy from the light source 332 to be directed tothe specimen without being contaminated or compromised by energy from anunwanted source. The light guard 350 may take almost any form. Thecamera 310 may be fastened to the spectrogram 330 or the camera may beheld in a housing or bracket (not illustrated.

Although the flash 312, flash detector 334, and light source 332 areillustrated in FIG. 23 as generating or detecting light energy,embodiments of the invention are not necessarily limited to generationof visible light to perform these functions. The flash 312 may be anelectromagnetic generator of any frequency, as may be the light source332. The detector 334 needs to be able to detect the energy generated bythe flash 312, of course. In some embodiments the flash 312 generates anelectromagnetic frequency that is different than an electromagneticfrequency of the energy source 332.

In some embodiments the energy source 332 is a laser and thespectrometer 330 is a Raman spectrometer. In some embodiments the energysource 332 is a broad-band infra-red source and the spectrometer 330 isan infra-red (IR) spectrometer. In some embodiments the energy source332 is a broad-band ultra-violet source and the spectrometer 330 is anultra-violet (UV) spectrometer. In some embodiments the energy source332 is a broad-band IR-VIS-UV source and the spectrometer 330 is anultra-violet (IR-VIS-UV) spectrometer. In some embodiments the energysource 332 is a broad-band terahertz source and the spectrometer 330 isa terahertz spectrometer. In some embodiments the energy source 332 isan electric arc or corona discharge source and the spectrometer 330 isan electric arc or corona discharge spectrometer, respectively.

Although specific embodiments of the invention have been illustrated anddescribed for purposes if illustration, it will be understood thatvarious modifications may be made without departing from the spirit andscope of the invention.

What is claimed is:
 1. A method of multichannel spectrogram imagerestoration of an image, the method comprising, for at least one of thechannels: detecting a first image patch as having a clipped portion;detecting a second image patch as having an unclipped portion;generating a scale factor minimizing a difference between scaledportions of the image near the second image patch, and correspondingportions of the image near the first clipped patch; and replacing theclipped portion of the first clipped image patch with a correspondingscaled portion of the second unclipped image patch.
 2. The method asrecited in claim 1 wherein, for each channel having a clipped portion inthe first image patch, the second image patch is in a same channel asand adjacent to the first image patch, along the primary spectrogramaxis.
 3. The method as recited in claim 1 wherein, for each channelhaving a clipped portion in the first image patch, the second imageincludes at least one other channel, and the second image patch isadjacent to the first image patch along the secondary spectrogram axis.4. The method as recited in claim 1, further comprising: copying a firstunclipped image patch that is adjacent to a second clipped image patch;selecting a third unclipped image patch that is adjacent the secondclipped image patch; selecting a forth unclipped image patch that isadjacent to the first unclipped image patch and having a same relativetwo-dimensional spatial offset as that between the third and secondimage patches. generating a scale factor as a ratio of the firstunclipped image patch and the forth unclipped image patch; generating afifth image patch by scaling the magnitudes of the first unclipped imagepatch by the scale factor; and replacing the second clipped image patchwith the fifth image patch.
 5. A method of converting a multichannelspectrogram image pixel to a wavelength measurement comprising:converting the spectrogram image pixel to color plane coordinates;projecting the color plane coordinates from an achromatic point in thecolor plane to coordinates of purest form in the color plane; andselecting a resultant wavelength that has color plane coordinatesclosest to an intersection of the projected color plane coordinates andthe coordinates of purest form in the color plane.
 6. The method asrecited in claim 5, further comprising refining the selection of theresultant wavelength by interpolating between wavelengths according tothe corresponding color coordinates of pure light known and thecoordinates of the pixel.
 7. The method as recited in claim 5, furthercomprising: converting a multichannel spectrogram image to magnitude by:converting each spectrogram pixel of the multichannel spectrogram imageto respective wavelength and magnitude using multiple detector channels,determining a theoretical spatial mapping of each wavelength, for eachtheoretical wavelength, collect measured wavelengths in the spatialvicinity, and determine the maximum respective magnitude among thecollection of measured wavelengths.
 8. The method as recited in claim 5,further comprising: converting a multichannel spectrogram image topurity measurements by: mapping each spectrogram pixel into coordinatesin a color plane, determining the coordinates of the respective purewavelength in the color plane, determining a first color plane distancebetween the achromatic point and the color plane coordinates for eachspectrogram pixel, determining a second color plane distance between theachromatic point and the color plane coordinates for each respectivepure wavelength in the color plane, and calculating the estimated purityas the first color plane distance divided by the second color planedistance.
 9. The method as recited in claim 8 wherein determining atheoretical spatial mapping comprises: selecting a first and a secondreference wavelengths as most likely to be accurately measured;predicting theoretical spatial mappings of all wavelengths; andmultiplexing between theoretical and measured wavelength according tothe purity magnitude.
 10. The method as recited in claim 9 whereinselecting two reference wavelengths comprises: identifying pixels withtwo channels having approximately equal magnitudes; and when a third ormore channel exists, the magnitudes of the remaining channels are lessthan the first two channels.
 11. The method as recited in claim 9,wherein the multiplexing is performed by degrees, and the resultingwavelength is a weighted sum of the theoretical and measuredwavelengths.
 12. A spectrometer system, comprising: a spectrogram forreading electromagnetic energy from a specimen; an energy source coupledto the spectrogram for directing electromagnetic energy to the specimen;a camera having a lens aligned to the spectrogram for reading an outputof the spectrogram, the camera further including a signal generator; anda signal detector coupled to the spectrogram and to the energy source,the signal detector structured to receive a signal from the signalgenerator of the camera and to generate the energy source in response toreceiving the signal.
 13. The spectrometer system of claim 1 in whichthe camera is fastened to the spectrogram.
 14. The spectrometer systemof claim 12 in which the signal generated by the signal generator has anelectromagnetic frequency that is different than an electromagneticfrequency of the energy source.
 15. The spectrometer system of claim 12in which the energy source is a laser for a Raman spectrometer.
 16. Thespectrometer system of claim 12 in which the energy source is abroad-band infra-red source for an infra-red (IR) spectrometer.
 17. Thespectrometer system of claim 12 in which the energy source is abroad-band ultra-violet source for an ultra-violet (UV) spectrometer.18. The spectrometer system of claim 12 in which the energy source is abroad-band IR-VIS-UV source for an ultra-violet (IR-VIS-UV)spectrometer.
 19. The spectrometer system of claim 12 in which theenergy source is a broad-band terahertz source for a terahertzspectrometer.
 20. The spectrometer system of claim 12 in which theenergy source is an electric arc or corona discharge source for anelectric arc or corona discharge spectrometer, respectively.